APPLICATION 
10 FOR 

UNITED STATES LETTERS PATENT 

Be it known that we, Dennis A. Dempsey, residing at Ballinveala, Crecora, Co. 

Limerick, Republic of Ireland, and Thomas G. O'Dwyer, residing at Shrawickeen, 

15 Clonlara, Co. Clare, Republic of Ireland, and Oliver James Brennan, residing at 14 Park 

Avenue, Adare, Co. Limerick, Republic of Ireland, and Alan Walsh, residing at Ballycon, 

Mountlucas, Tullamore, Co. Offaly, Republic of Ireland all being citizens of Ireland and 

Tudor Vinereanu, residing at 6 Keysers Court, Frenches Quay, Cork, Ireland and being a 

citizen of Romania have invented a certain new and useful 

20 INTEGRATED DIGITAL CALIBRATION CIRCUIT AND DIGITAL TO ANALOG 

CONVERTER (DAC) 

of which the following is a specification: 
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Applicant: 
For: 



Dempsey et al. 

INTEGRATED DIGITAL CALIBRATION CIRCUIT AND DIGITAL TO 
ANALOG CONVERTER (DAC) 



10 



FIELD OF THE INVENTION 
This invention relates to an integrated digital calibration circuit and digital to 
analog converter (DAC) for adjusting, modifying, modulating or correcting DAC transfer 
function end points. 



RELATED APPLICATIONS 
This application claims priority of U.S. Provisional serial no. 60/413,909, Dempsey 
et al. entitled DIGITAL-TO-ANALOG CONVERTER SYSTEM CALIBRATION, filed 
November 4, 2002 (AD-332J); and U.S. Provisional serial no. 60, 414,166, Dempsey et al. 
1 5 entitled DIGITAL-TO-ANALOG CONVERTER CALIBRATION ARCHITECTURE & 
SCHEME, filed September 27, 2002. 



BACKGROUND OF THE INVENTION 
Adjustment and control of DAC end points e.g. zero scale, full scale or gain and 
20 offset is desirable in a number of circumstances. One of the more important uses is end 

point error correction. That is correction of zero scale and full scale errors and gain and 
offset errors. 

Digital to analog converter (DAC) transfer function end point errors e.g. zero 
scale and full scale errors; offset and gain errors have historically been a problem 
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effecting (DAC) accuracy. Previous attempts to reduce such errors have included laser 
trimming which applied to resistors only and required special hardware and wafer 
fabrication techniques and added significantly to the cost. Mixed signal (analog/digital) 
calibration is another approach but its results can be risky, performance is limiting and 
5 less dependable. It too increases expense. One common approach is to design for the 
optimal nominal desired result. But the higher the accuracy desired the greater the 
expense: more precision design and production required, added software and/or circuitry 
and more expensive fabrication techniques. In addition, compensating for such errors in 
systems including the DACs, and their preceding reference path plus subsequent analog 
10 chain of circuits (i.e. analog signal chain) requires even greater complexity and expense 
either in additional and highly accurate integrated circuitry or calibrating system(s) 
procedures integrated into the system software. 



BRIEF SUMMARY OF THE INVENTION 
15 It is therefore an object of this invention to provide an improved integrated 

programmable digital calibration circuit and digital to analog converter (DAC) to adjust 
DAC transfer function end points. 

It is a further object of this invention to provide an improved integrated 
programmable digital calibration circuit and digital to analog converter (DAC) which 
20 provides accuracy and performance while maintaining lower cost and complexity. 

It is a further object of this invention to provide an improved integrated 
programmable digital calibration circuit which employs a wholly digital, less expensive 
approach. 
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It is a further object of this invention to provide an improved integrated 
programmable digital calibration circuit which can adjust end points not only in the DAC 
but in associated analog circuitry as well. 

It is a further object of this invention to provide an improved integrated 
5 programmable digital calibration circuit which reduces demands on external control 
circuits such as microcontrollers, DSPs and CPUs. 

It is a further object of this invention to provide an improved integrated 
programmable digital calibration circuit which can adjust zero scale and full scale and 
gain and offset. 

10 It is a further object of this invention to provide an improved integrated digital 

calibration circuit and digital to analog converter (DAC) to compensate for DAC transfer 
function end point error. 

It is a further object of this invention to provide an improved integrated digital 
calibration circuit and digital to analog converter (DAC) which provides greater accuracy 
15 and performance while maintaining lower cost and complexity. 

It is a further object of this invention to provide an improved integrated digital 
calibration circuit which employs a wholly digital less expensive approach. 

It is a further object of this invention to provide an improved integrated digital 
calibration circuit which can reduce errors not only in the DAC but in associated analog 
20 circuitry as well. 

It is a further object of this invention to provide an improved integrated digital 
calibration circuit which reduces demands on external control circuits such as 
microcontrollers, DSP's and CPU's. 
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It is a further object of this invention to provide an improved integrated digital 
calibration circuit which can correct zero scale and full scale errors and gain and offset 
errors. 

The invention results from the realization that a simple and inexpensive, but much 
5 more accurate, DAC can be achieved by integrating a calibration unit with the DAC to 

provide the DAC transfer function end point coefficients, e.g. gain and offset coefficients, 
zero scale and full scale coefficients, digitally, to the DAC, which coefficients can be 
stored externally or internally, on the calibration circuit or on a chip which holds both the 
calibration circuit and the DAC, and can be applied to adjust the DAC end points or the 

10 DAC end points and the end points of the analog circuits associated with the DAC and 
further the memory can be made user accessible for post fabrication error correction 
programming for the DAC and/or associated analog circuits. 

This invention features a programmable integrated digital calibration circuit and 
digital to analog converter including a digital to analog converter (DAC) and a digital 

1 5 calibration circuit including a memory for storing predetermined of the end point 

coefficients of the digital to analog converter transfer function and an arithmetic logic unit 
for applying the coefficients of the end points to the DAC input signal to adjust the end 
points of the DAC. 

In a preferred embodiment the end point coefficients may include the offset 
20 coefficient and gain coefficient. The arithmetic logic unit may include an arithmetic 

circuit having a multiplier circuit for multiplying the DAC input by the gain coefficient 
and an adder circuit for adding the offset coefficient to the input signal. The end point 
coefficients may include the zero scale and full scale coefficients. The arithmetic logic 
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unit may include an arithmetic circuit for algebraically using the zero scale output and 
ideal output and normalizing them in respect to the least significant bit (LSB) value to 
obtain the zero scale coefficient and combining the full scale output and ideal output and 
normalizing them in respect to the LSB to obtain the full scale coefficient and applying 
5 those coefficients to the input signal to the DAC. The digital calibration circuit and DAC 
may be on the same integrated circuit chip. The memory may be in the digital calibration 
circuit. The memory may be external to the digital calibration circuit. The memory may 
be a user accessible programmable memory. 

The invention also features a programmable digital calibration system including 

10 an integrated digital calibration circuit and digital to analog converter including a digital 
to analog converter (DAC) and an analog signal circuit responsive to the DAC. A digital 
calibration circuit includes a memory for storing the predetermined end point coefficient 
of the DAC transfer function and includes an arithmetic logic unit for applying the end 
point coefficients to the DAC input signal to compensate for the end point coefficients of 

15 the DAC and the analog signal circuit. 

In a preferred embodiment the digital calibration circuit and DAC may be on the 
same integrated circuit chip. The memory may be in the digital calibration circuit or may 
be external to the digital calibration circuit. The memory may be a user accessible 
programmable memory. 

20 The invention also features a programmable integrated digital calibration circuit 

and digital to analog converter including a digital to analog converter and a digital 
calibration circuit including a memory for storing the predetermined offset coefficient and 
gain coefficient of the digital to analog converter (DAC) and an arithmetic logic unit 
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including an arithmetic circuit having a multiplier circuit for multiplying the DAC input 
by the gain coefficient and an adder circuit for adding the offset coefficient to the input 
signal to adjust the gain and offset errors of the DAC. 

In a preferred embodiment the digital calibration circuit and DAC may be on the 
5 same integrated circuit chip. The memory may be in the digital calibration circuit or may 
be external to the digital calibration circuit. The memory may be a user accessible 
programmable memory. 

The invention also features a programmable digital calibration system including 
an integrated digital calibration circuit and digital to analog converter including a digital 
10 to analog converter (DAC) and an analog signal circuit responsive to the DAC. A digital 
calibration circuit includes a memory for storing the predetermined offset coefficient and 
gain coefficient of the DAC and the analog signal circuit and includes an arithmetic logic 
unit including an arithmetic circuit having a multiplier circuit for multiplying the DAC 
input by the gain coefficient and an adder circuit for adding the offset coefficient to the 
1 5 input signal to adjust the gain and offset of the DAC and the analog signal circuit. 

In preferred embodiments the digital calibration circuit and DAC may be on the 
same integrated circuit chip. The memory may be in the digital calibration circuit or may 
be external to it. The memory may be a user accessible programmable memory. 

The invention also features an integrated programmable digital calibration circuit 
20 and digital to analog converter including a digital to analog converter (DAC) and a digital 
calibration circuit including a memory for storing predetermined zero scale and full scale 
coefficients of the digital to analog converter DAC and an arithmetic logic unit including 
an arithmetic circuit for algebraically combining the zero scale output and ideal output 
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and normalizing them by the least significant bit (LSB) value to obtain the zero scale 
coefficient and combining the full scale output and ideal output and normalizing them by 
the LSB to obtain the full scale coefficient and applying those coefficients to the input 
signal to the DAC to adjust the zero scale and full scale of said DAC. 
5 In a preferred embodiment the digital calibration circuit and DAC are on the same 

integrated circuit chip. The memory may be in the digital calibration circuit or may be 
external to it and the memory may be user accessible programmable memory. 

The invention also features a programmable digital calibration system including 
an integrated digital calibration circuit and digital to analog converter including a digital 

10 to analog converter (DAC) and an analog signal circuit responsive to the DAC. A digital 
calibration circuit includes a memory for storing predetermined zero scale and full scale 
coefficients of the DAC and the analog signal circuit and an arithmetic logic unit 
including an arithmetic circuit for algebraically combining the zero scale output and ideal 
output and normalizing them by the least significant bit (LSB) value to obtain the zero 

15 scale coefficient and combining the full scale output and ideal output and normalizing 
them by the LSB to obtain the full scale error coefficient and applying those coefficients 
to the input signal to the DAC to adjust the zero scale and full scale offset of the DAC 
and the analog signal circuit. 

In a preferred embodiment the digital calibration circuit and DAC are on the same 

20 integrated circuit chip. The memory may be in the digital calibration circuit or external to 
it and the memory may be a user accessible programmable memory. 

This invention features an integrated digital calibration circuit and digital to analog 
converter including a digital to analog converter (DAC) and a digital calibration circuit 
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including a memory for storing predetermined complements of the end point errors of the 
digital to analog converter transfer function and an arithmetic logic unit for applying the 
complements of the end point errors to the DAC input signal to compensate for the end 
point errors of the DAC. 

5 In a preferred embodiment the complements of the end point errors may include 

the offset error and gain error. The arithmetic logic unit may include an arithmetic circuit 
having a multiplier circuit for multiplying the DAC input by the gain error coefficient and 
an adder circuit for adding the offset error coefficient to the input signal. The 
complements of the end point errors may include the zero scale and full scale error 

10 coefficients. The arithmetic logic unit may include an arithmetic circuit for algebraically 
counting the zero scale output and ideal output and normalizing them by the least 
significant bit (LSB) value to obtain the zero scale error and combining the full scale 
output and ideal output and normalizing them by the LSB to obtain the full scale error 
coefficient and applying those error coefficients to the input signal to the DAC, The 

15 digital calibration circuit and DAC may be on the same integrated circuit chip. The 

memory may be in the digital calibration circuit. The memory may be external to the 
digital calibration circuit. The memory may be a user accessible programmable memory. 

The invention also features a digital calibration system including an integrated 
digital calibration circuit and digital to analog converter including a digital to analog 

20 converter (DAC) and an analog signal circuit responsive to the DAC. A digital 

calibration circuit includes a memory for storing the predetermined complements of the 
end point error of the DAC transfer function and includes an arithmetic logic unit for 
applying the complements of the end point error to the DAC input signal to compensate 
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for the end point errors of the DAC and the analog signal circuit. 

In a preferred embodiment the digital calibration circuit and DAC may be on the 
same integrated circuit chip. The memory may be in the digital calibration circuit or may 
be external to the digital calibration circuit. The memory may be a user accessible 
5 programmable memory. 

The invention also features an integrated digital calibration circuit and digital to 
analog converter including a digital to analog converter and a digital calibration circuit 
including a memory for storing the predetermined offset error coefficient and gain error 
coefficient of the digital to analog converter (DAC) and an arithmetic logic unit including 
10 an arithmetic circuit having a multiplier circuit for multiplying the DAC input by the gain 
error coefficient and an adder circuit for adding the offset error coefficient to the input 
signal to compensate the gain and offset errors of the DAC. 

In a preferred embodiment the digital calibration circuit and DAC may be on the 
same integrated circuit chip. The memory may be in the digital calibration circuit or may 
15 be external to the digital calibration circuit. The memory may be a user accessible 
programmable memory. 

The invention also features a digital calibration system including an integrated 
digital calibration circuit and digital to analog converter including a digital to analog 
converter (DAC) and an analog signal circuit responsive to the DAC. A digital 
20 calibration circuit includes a memory for storing the predetermined offset error 

coefficient and gain error coefficient of the DAC and the analog signal circuit and 
includes an arithmetic logic unit including an arithmetic circuit having a multiplier circuit 
for multiplying the DAC input by the gain error coefficient and an adder circuit for 
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adding the offset error coefficient to the input signal to compensate the gain and offset 
error of the DAC and the analog signal circuit. 

In preferred embodiments the digital calibration circuit DAC may be on the same 
integrated circuit chip. The memory may be in the digital calibration circuit or may be 
5 external to it. The memory may be a user accessible programmable memory. 

The invention also features an integrated digital calibration circuit and digital to 
analog converter including a digital to analog converter (DAC) and a digital calibration 
circuit including a memory for storing predetermined zero scale and full scale error 
coefficients of the digital to analog converter DAC and an arithmetic logic unit including 
10 an arithmetic circuit for algebraically combining the zero scale output and ideal output 
and normalizing them by the least significant bit (LSB) value to obtain the zero scale 
error coefficient and combining the full scale output and ideal output and normalizing 
them by the LSB to obtain the full scale error coefficient and applying those error 
coefficients to the input signal to the DAC to compensate for the zero scale and full scale 
1 5 errors of said DAC. 

In a preferred embodiment the digital calibration circuit and DAC are on the same 
integrated circuit chip. The memory may be in the digital calibration circuit or may be 
external to it and the memory may be user accessible programmable memory. 

The invention also features a digital calibration system including an integrated 
20 digital calibration circuit and digital to analog converter including a digital to analog 
converter (DAC) and an analog signal circuit responsive to the DAC. A digital 
calibration circuit includes a memory for storing predetermined zero scale and full scale 
error coefficients of the DAC and the analog signal circuit and an arithmetic logic unit 
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including an arithmetic circuit for algebraically combining the zero scale output and ideal 
output and normalizing them by the least significant bit (LSB) value to obtain the zero 
scale error coefficient and combining the full scale output and ideal output and 
normalizing them by the LSB to obtain the full scale error coefficient and applying those 
5 error coefficients to the input signal to the DAC to compensate for the zero scale and full 

scale offset errors of the DAC and the analog signal circuit. 

In a preferred embodiment the digital calibration circuit and DAC are on the same 
integrated circuit chip. The memory may be in the digital calibration circuit or external to 
it and the memory may be a user accessible programmable memory. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 
Other objects, features and advantages will occur to those skilled in the art from 
the following description of a preferred embodiment and the accompanying drawings, in 
which: 

15 Fig. 1 is a graphical illustration of a digital to analog converter (DAC) transfer 

function showing end point errors as gain and offset errors; 

Fig. 2 is a graphic illustration of a digital to analog converter (DAC) transfer 
function showing end point errors as zero scale and full scale errors; 

Fig. 3 is a simplified schematic block diagram of an integrated digital calibration 
20 circuit and DAC employing gain and offset coefficients according to this invention. 

Figs. 4 A-C are more detailed schematic diagrams of a portion of three different 
embodiments of the digital calibration circuit for applying gain and offset coefficients to 
compensate for gain and offset errors; 
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Fig. 5 is a view similar to Fig. 3 of an integrated digital calibration circuit and 
DAC employing zero scale and full scale coefficients according to this invention; 

Fig. 6 is a view similar to Fig. 3 of an integrated digital calibration circuit and 
DAC employing a software driven calibration system according to this invention; and 
5 Fig. 7 is a flow chart of the software for operating the calibration system of Fig. 6. 

DISCLOSURE OF THE PREFERRED EMBODIMENT 
Aside from the preferred embodiment or embodiments disclosed below, this 
invention is capable of other embodiments and of being practiced or being carried out in 
10 various ways. Thus, it is to be understood that the invention is not limited in its 

application to the details of construction and the arrangements of components set forth in 
the following description or illustrated in the drawings. 

The invention contemplates the adjustment of DAC end points or DAC plus 
analog signal current chain end points for any reason of control or adjustment. However, 
15 one of the more important applications is to correct end point errors such as zero scale, 

full scale errors and gain and offset errors. This error correction application is the subject 
of the specific embodiment disclosed but is by no means a limitation of the scope of the 
invention which includes end point adjustment for any reason. 

Digital to analog converter (DAC) transfer function end point errors can be treated 
20 as gain and offset errors or as zero scale and full scale errors. There is shown in Figure 1 
a view of the gain and offset error approach. An ideal DAC transfer function 10 begins at 
the zero point, 12, and has a predetermined slope or gain. The actual transfer function 14 
would typically start at some offset 16 from zero introducing the offset error and will 
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have a slope or gain which is somewhat different than the slope or gain of the ideal 
transfer function 10. The deviation caused by the gain error in addition to the offset error 
combine to form the gain and offset error 18. Typically the coefficient to correct the 
offset error is referred to as c and the coefficient to correct the gain error is referred to as 
5 m. 

In another view, Fig. 2 the end point errors are viewed as zero scale errors 20 and 
full scale errors 22. The zero scale error occurs because the zero position, 23, of the 
transfer function 24 may not be at the ideal zero point, 26, and the full scale error occurs 
because the full scale point varies in dependence on errors that effect the gain or slope of 

10 the transfer function 24 and due to errors in the zero position, 23. 

In one embodiment according to this invention a DAC 30, Fig. 3, and digital 
calibration circuit 32 are integrated on a single chip 34. A calibration system 36 senses 
the output of DAC 30 at 38 or if desired the output from DAC 30 and the subsequent 
analog signal circuits or analog signal chain 40 at output 42. In this way the errors of the 

15 entire analog chain, including the DAC, can be compensated for. In either case, the 
signal is coupled via line 44 to calibration system 36 which delivers the offset error 
coefficient c and the gain error coefficient m to digital calibration circuit 32 where these 
values are used to calculate y where x is the input signal and y is the output signal. 
Simply stated 

20 y = mx + c (1) 

where y is the output, m is the gain or slope error coefficient, c is the offset coefficient 
and x is the input. 

Equation No. 1 is a classic, generic textbook linear algebraic expression, using 
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real numbers by default. In the case of a digital to analog converter the digital input of the 

DAC has finite, quantized resolution. Hence, quantized, finite resolution digital numbers 

are appropriate in the expressions for DACs. For this reason, we use a modified version 

of this expression in practice: 
5 y = (m+c2)/d) + c Eq (2) 

where: c2 is a second constant digital term and d is a divider factor used to scale down the 

output number. 

A specific, preferred embodiment is: 

y = ((m+l)*/2 A N)*x + c Eq (3) 
0 If the gain factor (m) has N effective bits of resolution, the maximum (or 

fullscale) m value = 2 A N-1. In equation (2), with fullscale m (m=2 A N-l), the effective 

gain = (2 A N-l+l)/2 A N= 1. 

The choice of factor in equation (2) is for implementation reasons. M = 2 A N- 

1 => gain = 1. Any digital constant can be used instead of +1. Similarly, the division by 
5 2 A N operation equates to positioning the taps from the digital multiplier output, as is 

known to those skilled in the art, and a different divider factor could also be chosen. 

For gains of greater than unity, the range of m+1 is greater than the divider factor 

d. 

The values of m and c^ may be stored in a memory 50 which may be on chip 34 
) but external to digital calibration circuit 32, or they may be contained within digital 
calibration circuit 32 memory 50' or may be external to both DAC 30 and digital 
calibration circuit 32 and chip 34 as shown by memory 50". When memory 50" is 
provided external to chip 34 it may be made accessible and programmable by the user so 
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that not just the input signal x but the offset coefficient c and gain coefficient m may be 
chosen to compensate for DAC errors or for DAC errors and errors contributed by any 
one or more of the subsequent analog signal circuits in chain 40. 

Included in digital calibration circuit 32 is an arithmetic logic unit 33, such as 
5 arithmetic circuit 52, Fig. 3, which includes a multiplier 54 and adder 56, Fig. 4. 

Arithmetic circuit 52 thus performs the equation y = mx + c. It receives the x input to 
multiplier 54 where it is multiplied by the m coefficient to which is added the c 
coefficient in adder 56 to produce the output y. Alternatively, arithmetic unit 32a, Fig. 

4B, may include an additional divider 55 to obtain the output y = — + c . That is, divider 

d 

1 0 55 divided by dthe product p_ of the DAC input x and gain offset m to obtain a quotient q 
which is then added to the offset c by adder 56 where d is a divider factor used to scale 
down the output number. Or as shown at 52b, Fig. 4C, there may be an adder 57 which 

combines coefficient c2 59 to obtain the output y = ( w + c2 )** + c That igj b e f ore the 

d 

gain offset m is multiplied by the DAC input x to obtain product p, the gain offset is 

1 5 combined with a second constant c2, 59. 

Alternatively, as shown in Fig. 5, the arithmetic circuit 52a in the arithmetic logic 
unit 33a in digital calibration circuit 32a may produce a zero scale error coefficient, zs, 
and the full scale coefficient, fs, or may provide them along with input x to digital 
calibration circuit 32a. In the same way as previously, memory 50a, 50'a, and 50"a may 

20 perform in a similar fashion but with respect to full scale coefficient fs and zero scale 

coefficient zs. Calibration system 36 may include hardware circuits or a combination of 
hardware circuits and software for sensing the zero scale and full scale outputs in 
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calculating the zs and fs coefficients or the m and ^coefficients. Calibration system 36b, 
Fig. 6, would contain an ADC at its input externally, ADC 37 or internally 37', for 
converting analog signal circuit chain at 42b to digital format for use in arithmetic 
operations to calculate the coefficients. Analog signal circuit (chain) 40b may also 
5 include V re f+ circuit 5 1 and V re f- circuit 53. It may also be done entirely in software in 
calibration system 36b, Fig. 6, in three phases, 1,2, and 3 which may calculate either the 
zs and fs coefficients or the m and c coefficients as shown in the software flowcharts of 
Fig. 7. 

The calibration cycle starts with setting the DAC input at zero, step 100, Fig. 7, 
1 0 and then measuring the voltage/current output, in step 1 02, of the DAC or of the analog 
signal circuit chain 40. Then the DAC is set to full scale in step 104 and again the output 
is measured. In step 106 the output of either DAC 30 or the analog signal circuit chain 40 
is used to calculate the coefficients m and c. The LSB size is calculated in step 108. The 
c/zs terms are calculated in step 1 10 after which m, step 1 12 and dfs, step 1 14 are 
1 5 calculated. Next c/zs and m step 1 1 6 and c/zs and fs, step 1 1 8 are quantized, fs is 
calculated, step 120 and c/zs and fs are quantized, 122. 

To correct for end point error using the zero scale zs coefficient and the full scale 
fs coefficient the coefficient zs can be obtained arithmetically by algebraically summing 
the voltage out at zero scale minus the voltage out at zero scale ideal and normalizing that 
20 or dividing it by the least significant bit size. 

The full scale correction coefficient fs can be obtained in by algebraically 
summing the V out full scale minus the V out full scale ideal and normalizing that with the 
least significant bit size. 
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Note that the calibration system 36 can include software, a DSP or a 
microcontroller or other device to effect the determination of m and c or zs and fs. 

Example 1: Calculates m and c Coefficients to Correct Gain and Offset Errors. 

5 Vref+ = 4.096V 

V re f- = 0V 

DAC resolution, N = 12 

Therefore, ideal LSB = — - = \ mv 

2" 

Step 100 => 102 => Voutzs = -lOmv 
10 Step 100 x 106 => Voutfs = 4.1 15v 

New Step: Calculate LSB size = V "*~ V «*> = 4.115-0.01 = i.002442mv 

2" -1 4095 

Calculates c/zs = = -(-0-01-0) = 991$6 

LSBsize LSBsize 

Quantize c/zs to required resolution (12b) => 10 

n , 1+ Ideal Output Range 4.095 

Calculate m = — = = 0.997564 

y^-y ouas 4.115-0.010 

15 Quantize m to required level (12b) => 4086 

Example 2: Calculating zs and fs Coefficients to Cancel Zero Scale and Full Scale 
Errors. 

We can extend example No. 1 and use the same data to calculate new coefficients 
as follows: 

20 Set dfs = = -(4.H5V-4.096) = { 

LSBsize 1.002442mv 
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= 18.954 LSBs 
Quantizing dfs to 12b (LSB) level: dfs -> 19 LSBs 

Set fs = = 4 ' 115 = 4104.97565 

LSBsize 1.002442mv 

Quantizing fs to 12b (LSB) level: fs -> 4105 
5 Although specific features of the invention are shown in some drawings and not in 

others, this is for convenience only as each feature may be combined with any or all of the 
other features in accordance with the invention. The words "including", "comprising", 
"having", and "with" as used herein are to be interpreted broadly and comprehensively 
and are not limited to any physical interconnection. Moreover, any embodiments 
1 0 disclosed in the subject application are not to be taken as the only possible embodiments. 
Other embodiments will occur to those skilled in the art and are within the 
following claims: 

What is claimed is: 



AD-332J/APD2332-1-US 
JSI/dmg 



19 



